Producing and verifying computational determinations using a distributed ledger

ABSTRACT

Software for producing and verifying computational determinations using a distributed ledger, by: (i) receiving a first input from a user; (ii) producing a first computational determination utilizing a first computational model, wherein the first computational determination includes a first computational output that is based, at least in part, on the first input; (iii) computing a hash of the first computational model; (iv) sending a record of the first computational determination to a verification system, wherein the record of the first computational determination includes the hash of the first computational model; (v) receiving a verification from the verification system indicating that the hash of the first computational model matches a hash of a second computational model and that the record of the first computational determination has been stored in a first distributed ledger; and (vi) in response to receiving the verification, providing the first computational output to the user.

BACKGROUND

The present invention relates generally to the field of distributedledger-based systems, and more particularly to systems for producing andverifying computational determinations using a distributed ledger.

Blockchain refers to a distributed, permissioned, and immutable ledgercapable of recording transactions. Blockchain is a decentralizedtechnology consisting of a peer-to-peer (P2P) network includingcomputers referred to as nodes. Blockchain further includes methods forthe nodes to validate transactions. Once a transaction is validated bythe nodes, a new block is added to the existing blockchain containinginformation to confirm the transaction.

Artificial Intelligence (AI) is broad field of computer science thatgenerally applies to situations where machines mimic cognitive functionsthat humans associate with human minds. Machine Learning (ML) is a fieldwithin AI that refers to the ability of a computer to learn and makepredictions instead of relying on static program instructions. Machinelearning is often employed for tasks where designing and implementingstatic program instructions with good performance is difficult orinfeasible, such as in email filtering, detection of network intruders,and computer vision.

SUMMARY

According to an aspect of the present invention, there is a method,computer program product, and/or computer system that performs thefollowing operations (not necessarily in the following order): (i)receiving a first input from a user; (ii) producing a firstcomputational determination utilizing a first computational modeldeployed in a production environment, wherein the first computationaldetermination includes a first computational output that is based, atleast in part, on the first input; (iii) computing a hash of the firstcomputational model using a cryptographic hash function; (iv) sending arecord of the first computational determination to a verificationsystem, wherein the record of the first computational determinationincludes the hash of the first computational model; (v) receiving averification from the verification system indicating that the hash ofthe first computational model matches a hash of a second computationalmodel and that the record of the first computational determination hasbeen stored in a first distributed ledger; and (vi) in response toreceiving the verification, providing the first computational output tothe user.

BRIEF DESCRIPTION OF THE DRAWINGS

To easily identify the discussion of any particular element or act, themost significant digit or digits in a reference number refer to thefigure number in which that element is first introduced.

FIG. 1 is a block diagram depicting networked computers system 100,according to an embodiment of the present invention.

FIG. 2 shows flowchart 200 depicting a method to immutably store acomputational model and a record of a computational determination madeusing the computational model, according to an embodiment of the presentinvention.

FIG. 3 shows flowchart 300 depicting a method to produce, verify, andprovide a computational determination to a user, according to anembodiment of the present invention.

FIG. 4 is a block diagram depicting program 400, according to anembodiment of the present invention.

FIG. 5 illustrates computational determination environment 500, inaccordance with an embodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention recognize that ArtificialIntelligence (AI) and Machine Learning (ML) are prevalent throughoutmany product domains, and that the models built using these techniqueshave a vast influence on end users. For example, in some cases ML modelsare used for loan approval and insurance premium estimation. In thesecases, it can be imperative—for ethical, regulatory, and potentiallylegal reasons—to be able to explain why particular predictions have beenmade by ML models.

Embodiments of the present invention solve these problems usingdistributed ledger (e.g., blockchain) technology. In an embodiment, twoblockchains are used: one to store machine learning models and the otherto store model inferencing events. In this embodiment, by usingblockchain technology, the history of models used along with anyinferences delivered by those models are immutably stored, therebyallowing guaranteed AI traceability as required by policy and/orregulation.

For example, if a company is alleged to have used an ML model in anon-permitted way, embodiments of the present invention can retrieve,verify, and present the correct associated model and inferencing eventsused by the company in order to determine whether the use was compliantwith regulations. And by storing both the model and its correspondinginferencing events, embodiments of the present invention can help todetermine whether bias is present in either the model itself or one ofits inferencing events. Furthermore, by only requiring the storing ofthe model and its inferencing events, and not all of a user's data,embodiments of the present invention can reduce the amount of userinformation required to perform the above-described determinations.

In some embodiments, AI models deployed to production are themselvesstored on a blockchain, including preprocessing code/parameters, modelcode/parameters, and associated metadata such as time of deployment. Themodels can then be represented as unique hashes.

In these embodiments, when a model deployed to production performsinferencing when interacting with a user (for example, when a usersubmits a loan application and the model determines whether the user isapproved) the associated data and the result of that inferencing eventare stored on the second blockchain along with the model's hash from thefirst blockchain. Associated data generally includes the data input intothe model, and may include data or feature sets provided from the useror retrieved from other locations. The associated data, inferenceresult, and model hash are only stored on the second blockchain if themodel hash can be found on the first chain and the inference result isverified using the data and model.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general-purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

An embodiment of a possible hardware and software environment forsoftware and/or methods according to the present invention will now bedescribed in detail with reference to the Figures. FIG. 1 is afunctional block diagram illustrating various portions of networkedcomputers system 100, including: server sub-system 102; clientsub-system 104; client sub-system 106; client sub-system 108, clientsub-system 110, client sub-system 112; communication network 114; servercomputer 116; communication unit 118; processor set 120; input/output(I/O) interface set 122; memory device 124; persistent storage device126; display device 132; external device set 134; random access memory(RAM) devices 128; cache memory device 130; and program 400.

Sub-system 102, in many respects, representative of the various computersub-system(s) in the present invention. Accordingly, several portions ofsub-system 102 will now be discussed in the following paragraphs.

Sub-system 102 may be a laptop computer, tablet computer, netbookcomputer, personal computer (PC), a desktop computer, a personal digitalassistant (PDA), a smart phone, or any programmable electronic devicecapable of communicating with the client sub-systems via network 114.Program 400 is a collection of machine readable instructions and/or datathat is used to create, manage and control certain software functionsthat will be discussed in detail, below.

Sub-system 102 is capable of communicating with other computersub-systems via network 114. Network 114 can be, for example, a localarea network (LAN), a wide area network (WAN) such as the Internet, or acombination of the two, and can include wired, wireless, or fiber opticconnections. In general, network 114 can be any combination ofconnections and protocols that will support communications betweenserver and client sub-systems.

Sub-system 102 is shown as a block diagram with many double arrows.These double arrows (no separate reference numerals) represent acommunications fabric, which provides communications between variouscomponents of sub-system 102. This communications fabric can beimplemented with any architecture designed for passing data and/orcontrol information between processors (such as microprocessors,communications and network processors, etc.), system memory, peripheraldevices, and any other hardware components within a system. For example,the communications fabric can be implemented, at least in part, with oneor more buses.

Memory device 124 and persistent storage device 126 arecomputer-readable storage media. In general, memory device 124 caninclude any suitable volatile or non-volatile computer-readable storagemedia. It is further noted that, now and/or in the near future: (i)external device set 134 may be able to supply, some or all, memory forsub-system 102; and/or (ii) devices external to sub-system 102 may beable to provide memory for sub-system 102.

Program 400 may include a distributed, permissioned, and immutableledger such as a blockchain. In some embodiments, the blockchainconsists of multiple nodes that communicate with each other. In someembodiments, the blockchain may include nodes of three types: (i) clientnodes that submit transaction-invocations, (ii) peer nodes that committransactions and maintain the state and a copy of the ledger, and (iii)orderer nodes that implement communication services with deliveryguarantees, such as atomic or total order broadcast.

Program 400 may also include one or more consensus mechanisms forvalidating transactions before they are permanently stored on a block ofthe blockchain. In some embodiments of the present invention, consensusis the verification of the correctness of one or more transactionscomprising a block. Consensus may be achieved when the order and resultsof one or more transactions in a block have met the explicit policycriteria checks. Consensus methods may include endorsement policies todictate which specific members of a permissioned blockchain network mustendorse a certain transaction class. Consensus methods may furtheremploy chaincode (e.g., a prescribed interface for a business logicagreed to by members of the permissioned blockchain) to ensure that theendorsement policies are enforced by, for example, verifying that enoughendorsements are present and/or verifying that the endorsements derivedfrom the appropriate members of the permissioned blockchain network.After verifying the appropriate endorsements are present, a versioningcheck may include an agreement or consent of the current state of theledger before any blocks containing transactions are appended to theledger.

Program 400 may also include programs called chaincode, hold state andledger data, and execute transactions. Program 400 chaincode may be usedfor executing smart contracts (e.g., automatically executingtransactions and recording information onto the ledger) based on eventsoccurring on the blockchain. Smart contracts may also includemutually-agreed conditions by members of a permissioned blockchainnetwork for transactions to take place. The chaincode may be the centralelement of the blockchain as transactions are operations invoked on thechaincode. In some embodiments, transactions may have to be “endorsed”and only endorsed transactions may be committed and have an effect onthe state. In some embodiments, the blockchain may include one or morespecial chaincodes for management functions and parameters, collectivelycalled system chaincodes. In an exemplary embodiment, program 400 may beimplemented in a framework such as Hyperledger Fabric. (Note: theterm(s) “HYPERLEDGER” and/or “HYPERLEDGER FABRIC” may be subject totrademark rights in various jurisdictions throughout the world and areused here only in reference to the products or services properlydenominated by the marks to the extent that such trademark rights mayexist.)

Program 400 is stored in persistent storage device 126 for access and/orexecution by one or more of the respective computer processors ofprocessor set 120, usually through one or more memories of memory device124. Persistent storage device 126: (i) is at least more persistent thana signal in transit; (ii) stores the program (including its soft logicand/or data), on a tangible medium (such as magnetic or opticaldomains); and (iii) is substantially less persistent than permanentstorage. Alternatively, data storage may be more persistent and/orpermanent than the type of storage provided by persistent storage device126. Program 400 may also be stored and accessed from a public orprivate cloud service (e.g., blockchain-as-a-service). program 400 mayinclude both machine readable and performable instructions and/orsubstantive data (that is, the type of data stored in a database). Inthis particular embodiment, persistent storage device 126 includes amagnetic hard disk drive. To name some possible variations, persistentstorage device 126 may include a solid state hard drive, a semiconductorstorage device, read-only memory (ROM), erasable programmable read-onlymemory (EPROM), flash memory, or any other computer-readable storagemedia that is capable of storing program instructions or digitalinformation.

The media used by persistent storage device 126 may also be removable.For example, a removable hard drive may be used for persistent storagedevice 126. Other examples include optical and magnetic disks, thumbdrives, and smart cards that are inserted into a drive for transfer ontoanother computer-readable storage medium that is also part of persistentstorage device 126.

Communication unit 118, in these examples, provides for communicationswith other data processing systems or devices external to sub-system102. In these examples, communication unit 118 includes one or morenetwork interface cards. Communication unit 118 may providecommunications through the use of either or both physical and wirelesscommunications links. Any software modules discussed herein may bedownloaded to a persistent storage device (such as persistent storagedevice 126) through a communications unit (such as communication unit118).

I/O interface set 122 allows for input and output of data with otherdevices that may be connected locally in data communication with servercomputer 116. For example, I/O interface set 122 provides a connectionto external device set 134. External device set 134 will typicallyinclude devices such as a keyboard, keypad, a touch screen, and/or someother suitable input device. External device set 134 can also includeportable computer-readable storage media such as, for example, thumbdrives, portable optical or magnetic disks, and memory cards. Softwareand data used to practice embodiments of the present invention, forexample, program 400, can be stored on such portable computer-readablestorage media. In these embodiments the relevant software may (or maynot) be loaded, in whole or in part, onto persistent storage device 126via I/O interface set 122. I/O interface set 122 also connects in datacommunication with display device 132.

Display device 132 provides a mechanism to display data to a user andmay be, for example, a computer monitor or a smart phone display screen.

The programs described herein are identified based upon the applicationfor which they are implemented in a specific embodiment of theinvention. However, it should be appreciated that any particular programnomenclature herein is used merely for convenience, and thus theinvention should not be limited to use solely in any specificapplication identified and/or implied by such nomenclature.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

FIG. 2 shows flowchart 200 depicting a method according to the presentinvention. FIG. 3 shows flowchart 300 depicting another method accordingto the present invention. FIG. 4 shows program 400 for performing atleast some of the method operations of flowchart 200 and flowchart 300.The methods of flowchart 200 and flowchart 300, and the associatedsoftware of program 400, will now be discussed over the course of thefollowing paragraphs, with reference to FIG. 2 and FIG. 3 (for themethod operation blocks) and FIG. 4 (for the software module “mod”blocks).

The following paragraphs will also refer extensively to a simple exampleembodiment (also referred to as the “present example,” the “presentexample embodiment,” and the like). In this example embodiment, program400 is a program with two primary modules (“mods”): blockchain mod 402,for storing information on a blockchain, and computationaldetermination/validation mod 404, for performing computationaldeterminations and providing computational determination output to auser. It should be noted that this example embodiment is describedherein for example purposes, in order to help depict the scope of thepresent invention. As such, other embodiments may be configured indifferent ways or refer to other features, advantages, and/orcharacteristics not fully discussed herein.

Referring to flowchart 200 (see FIG. 2), in operation 202, blockchainmod 402 (see FIG. 4) receives an indication that a first computationalmodel is ready to be deployed into a production environment. Acomputational model, as described herein, is simply a computer programthat has been programmed to make decisions (also referred to as“computational determinations”) that generally include a set of outputs(also referred to as “computational model outputs” or “computationaloutputs”) based on a set of inputs (also referred to as “computationalmodel inputs” or “inputs”). In many cases, computational models are usedto represent—and therefore predict—activity that happens in the “real”(i.e., non-digital) world. For example, computational models can includemodels for forecasting weather, models for protein folding, and evenmodels for representing or mimicking features of the human brain (forexample, neural network models).

In certain embodiments, the first computational model is a MachineLearning (“ML”) model. Generally speaking, a ML model is a model thatcan learn from and make predictions on data, typically by ingestingsample sets of training data. ML models can be trained using supervisedlearning, which includes pre-labelled data, as well as usingsemi-supervised or unsupervised learning, which instead rely on findinghidden patterns in unlabeled or underlabeled data. A difference betweenML models and certain other types of computational models is that,because ML models are created using training data instead of by writingstatic algorithms, it can be difficult to determine how a ML model madea particular determination. Furthermore, because ML models have thecapacity to learn from their own decisions, it can also be difficult todetermine the particular state a ML model was in when it made a givendetermination. As such, when auditing the computational determinationsperformed by an ML model, it can be important to have copies of not onlyeach computational determination that was made, but also of eachcomputational model that was used to make each respective determination.

In the present example embodiment, the first computational model is a MLmodel that has been trained to identify domestic cats (Felis catus) inimages. That is, in this embodiment, the first computational modelreceives images as input, and in response it produces firstcomputational model output in the form of a Boolean determination, withTRUE indicating that an image includes a cat, and with FALSE indicatingthat an image does not include a cat. In operation 202, blockchain mod402 receives an indication that the ML model for identifying cats isready to be deployed in a production environment, so that the model canbe used to identify cats in user-submitted images.

In operation 204, blockchain mod 402 stores a copy of the firstcomputational model in a first distributed ledger. By storing the copyof the first computational model in the first distributed ledger,blockchain mod 402 creates an immutable record of the firstcomputational model prior to deployment to production. Becausedistributed ledgers are designed to store information in a verifiableand permanent way, storing the first computational model itself in adistributed ledger—such as the first distributed ledger—results in apermanent copy of a version of the first computational model prior todeployment.

The first distributed ledger may be distributed across a variety ofclient computing devices, or “nodes”, including, for example, clientsub-system 104, client sub-system 106, client sub-system 108, clientsub-system 110, and client sub-system 112 of networked computers system100. The nodes may be controlled by a single entity, such as the companythat controls the production environment, or by multiple entities,including, for example, independent third parties and/or regulatoryauthorities. Consensus for storing the copy of the first computationalmodel in the first distributed ledger may be achieved by any known (oryet to be known) consensus mechanism, including the consensus mechanismsdescribed above.

In some embodiments, including the present example embodiment, the firstdistributed ledger is a blockchain, and storing the first computationalmodel in the first distributed ledger includes storing the firstcomputational model in a block of the blockchain.

In some embodiments, including the present example embodiment, storingthe copy of the first computational model in the first distributedledger includes storing, in the first distributed ledger, one or more ofthe following: preprocessing code, preprocessing parameters, model code,model parameters, and/or metadata of the first computational model. Insome embodiments, these items are stored in a single entry of thedistributed ledger (for example, a block of a blockchain), and in otherembodiments, these items are stored in separate entries that are linkedsuch that they can be identified together at a later time (for example,as consecutive blocks in a blockchain).

The preprocessing code and parameters may include any combination ofcode and parameters needed to prepare inputs for processing by the firstcomputational model. For example, the preprocessing code may be codethat converts input into a particular file type or format, and thepreprocessing parameters may specify the desired dimensions, metadata,or file size limitations for the conversion. In the present exampleembodiment, for example, the preprocessing code converts received imagesinto the JPEG format, and the preprocessing parameters define that theconverted JPEG images should have a resulting resolution of 1536×1024pixels. Of course, in other embodiments, other combinations ofpreprocessing code and/or parameters may be used.

The model code and parameters may include the first computational modelitself as well as any information needed for the first computationalmodel to produce computational model output from an input or set ofinputs. The model code may be in the form of uncompiled source code,such that each line of code is viewable, or the model code may becompiled object code, such that the code itself is hidden but isnonetheless operable to produce computational model output from an inputor set of inputs. In the present example embodiment, for example, thecompiled object code of the ML model is used, as the source code wouldlikely not be particularly helpful in determining how the ML modelperforms its computational determinations. Furthermore, the presentexample embodiment stores as a model parameter a desired confidencelevel for outputting a TRUE result. In other words, because the ML modelgenerates a Boolean output (TRUE or FALSE), the ML model includes aconfigurable parameter that indicates a level of confidence at which theML model needs to indicate that an image contains a cat in order for theresult to be TRUE. For example, a model parameter of 0.8 would result inthe ML model outputting TRUE when it determines that an image includes acat with at least 80% confidence. Of course, in other embodiments, othercombinations of model code and/or parameters may be used.

The metadata of the first computational model may be any metadata thatmay be helpful in identifying the first computational model or otherwiseassisting a reviewer/auditor who is reviewing the first computationalmodel at a later time. For example, in some embodiments, the metadata ofthe first computational model includes a time representing thedeployment of the first computational model into the productionenvironment. In the present example embodiment, for example, the timerepresenting the deployment of the first computational model into theproduction environment is the time that the indication that the firstcomputational model is ready to be deployed into the productionenvironment was received by blockchain mod 402.

In operation 206, blockchain mod 402 computes a hash of the firstcomputational model using a cryptographic hash function. Generallyspeaking, a purpose of the hash is to create a unique identifier for thefirst computational model in a manner that is cryptographicallydifficult to duplicate without having an actual copy of the firstcomputational model, thereby creating a secure way to identify the firstcomputational model. Many known (or yet to be known) cryptographic hashfunctions may be used, including, but certainly not limited to, MD5,SHA-1, SHA-2, SHA-3, RIPEMD-160, and the like. Furthermore, manydifferent features of the first computational model may be used asinputs for computing the hash, including, but not limited to: (i) modelcode, (ii) model parameters, (iii) preprocessing code, (iv)preprocessing parameters, (v) metadata of the first computational model,and (vi) combinations thereof. In the present example embodiment, thehash of the first computational model is created by hashing the compiledobject code of the first computational model using an MD5 hashingalgorithm. In this embodiment, the computed hash is stored in theblockchain along with the information discussed above.

In operation 208, blockchain mod 402 sends a verification that the firstcomputational model has been successfully stored in the firstdistributed ledger. Generally speaking, the verification is sent toindicate that the first computational model is ready to be deployed inthe production environment, and as such may be sent to a deploymentenvironment such as deployment environment 504, discussed below. Bymaking the deployment of the first computational model into productionconditional on the storing of the first computational model in the firstdistributed ledger, program 400, and networked computers system 100 as awhole, can ensure that the first computational model has been immutablystored prior to using the first computational on any live productiondata. In the present example embodiment, for example, once blockchainmod 402 sends the verification that the first computational model hasbeen successfully stored in the first distributed ledger, the ML modelis deployed into production.

In operation 210, blockchain mod 402 receives an indication that asecond computational model has been used in the production environmentto produce a first computational determination. In other words, inoperation 210, blockchain mod 402 receives an indication (for example,from computational determination/validation mod 404) that a firstcomputational determination has been made in the production environment.It should be noted that, at this point, the identity of the secondcomputational model has not yet been determined, and blockchain mod 402is simply receiving notice that a computational determination has beenmade so that further action can be taken. In the present exampleembodiment, for example, not only does the previously described ML modelreside in the production environment, but so do a set of other relatedML models, including other ML models for identifying cats within imagesthat are trained using slightly different methods. While blockchain mod402 will ultimately be used to determine whether the secondcomputational model is the same as the first computational model, at thetime that blockchain mod 402 receives the indication in operation 210,blockchain mod 402 does not yet have enough information to determinewhether the computational determination has been made by the ML modelfor identifying cats in images or by one of these other ML models.

In operation 212, blockchain mod 402 receives a hash of the secondcomputational model, wherein the hash of the second computational modelis computed using the cryptographic hash function. In this operation,instead of performing the hash itself, blockchain mod 402 receives thehash from computational determination/validation mod 404, and will usethe hash in the next operation (operation 214) to verify the identity ofthe second computational model. In other embodiments, the computation ofthis hash is instead performed by blockchain mod 402. However, it shouldbe noted that in this particular embodiment, in order to ensure dataintegrity, it is important for the hashes in operation 212 and operation206 to be computed using information from separate respective locations,with operation 212 involving a computational model deployed in theproduction environment and with operation 206 involving a computationalmodel that has not yet been deployed.

In operation 214, blockchain mod 402, in response to determining thatthe hash of the second computational model matches the hash of the firstcomputational model, stores a record of the first computationaldetermination in a second distributed ledger, where the record of thefirst computational determination identifies the second computationalmodel as being the first computational model and includes the hash ofthe first computational model. That is, in operation 214, blockchain mod402 determines that the second computational model and the firstcomputational model are, in fact, the same computational model, andaccordingly stores a record of the first computational determination ina second distributed ledger along with an identifier to show that thefirst computational model was used to make the first computationaldetermination. As such, networked computers system 100 now has twoseparate, immutable ledgers each storing important components of thecomputational determination process: the actual computationaldeterminations that were made, and the computational models used to makethem. And by using a hash of a computational model prior to deploymentand again during production, networked computers system 100 can verifyand immutably record the exact version of the computational model thatwas used for each computational determination.

As with the first distributed ledger, the second distributed ledger maybe distributed across a variety of client computing devices, or “nodes”,including, for example, client sub-system 104, client sub-system 106,client sub-system 108, client sub-system 110, and client sub-system 112of networked computers system 100. The nodes may be controlled by asingle entity, such as the company that controls the productionenvironment, or by multiple entities, including, for example,independent third parties and/or regulatory authorities. Consensus forstoring the record of the first computational determination in thesecond distributed ledger may be achieved by any known (or yet to beknown) consensus mechanism, including the consensus mechanisms describedabove.

In some embodiments, including the present example embodiment, thesecond distributed ledger is a blockchain, different from the firstdistributed ledger's blockchain. In these embodiments, storing therecord of the first computational determination in the seconddistributed ledger includes storing the record in a block of the seconddistributed ledger's blockchain.

The record of the first computational determination that is stored inthe second distributed ledger may comprise any of a wide variety ofinformation about the first computational determination that may behelpful to a reviewer/auditor who is reviewing the first computationaldetermination at a later time. For example, in some embodiments, therecord of the first computational determination includes the firstcomputational model output and the first computational model input. Inthe present example embodiment, for example, this includes an inputimage along with the Boolean value of TRUE or FALSE indicating whetherthe input image includes a cat. In some embodiments, the record isstored in a single entry of the second distributed ledger (for example,a block of a blockchain), and in other embodiments, the different itemsof the record are stored in separate entries that are linked such thatthey can be identified together at a later time (for example, asconsecutive blocks in a blockchain).

While the foregoing description and examples relating to FIG. 2 describethe process for a single computational model and a single computationaldetermination to be stored in respective distributed ledgers,embodiments of the present invention recognize that the respectivedistributed ledgers are also capable of storing multiple computationalmodels and/or computational determinations for later analysis. Forexample, in an embodiment, blockchain mod 402 receives an indicationthat a third computational model is ready to be deployed into theproduction environment. In this embodiment, blockchain mod 402 thenstores the third computational model in the first distributed ledger ina ledger entry that succeeds the entry where the first computationalmodel is stored. Furthermore, blockchain mod 402 may also, when storingthe third computational model, include the hash of the firstcomputational model in the same ledger entry as the third computationalmodel, such as in a block in a blockchain (where blockchain blocksgenerally include a cryptographic hash of the previous block).Similarly, in another (or the same) embodiment, blockchain mod 402 mayreceive an indication that a fourth computational model has been used inthe production environment to produce a second computationaldetermination. In this embodiment, blockchain mod 402 may then receive ahash of the fourth computational model and, in response to determiningthat the hash of the fourth computational model matches either the hashof the first computational model or a hash of the third computationalmodel, store a record of the second computational determination in thesecond distributed ledger, wherein the record of the secondcomputational determination identifies the fourth computational model asbeing either the first computational model or the third computationalmodel, and includes the hash of the fourth computational model or thehash of the second computational model, such as in a block in ablockchain.

Referring now to flowchart 300 (see FIG. 3), in operation 302,computational determination/validation mod 404 receives a first inputfrom a user. Generally speaking, the first input is input for which theuser desires a computational determination to be made by a computationalmodel. The type and form of the input can vary, depending on theparticular requirements of a given computational model. For example, insome embodiments, the input is numeric or alphanumeric. In otherembodiments, the input is a file type; for example, in the presentexample embodiment, the input is an image file. In still otherembodiments, the input is a complex combination of values; for example,when the computational model is a weather prediction model, the inputscan include large quantities of current and past observational weatherdata (for example, temperature, precipitation, visibility, etc.).

In operation 304, computational determination/validation mod 404produces the first computational determination utilizing the secondcomputational model deployed in the production environment. As mentionedabove, the first computational determination includes the firstcomputational output that is based, at least in part, on the firstinput. While in the current example embodiment, the first computationaloutput includes a Boolean value of TRUE or FALSE, the firstcomputational output can be any kind of output that the secondcomputational model is capable of producing. For example, in someembodiments, the first computational output is numeric or alphanumeric.In other embodiments, the first computational output is a file—such asan image file, an audio file, or a video file. In still otherembodiments, the first computational output is a complex combination ofvalues; for example, when the second computational model is a weatherprediction model, the first computational output includes a completeweather forecast comprising a plurality of values for a plurality oflocations.

In operation 306, computational determination/validation mod 404computes a hash of the second computational model using thecryptographic hash function, using the same inputs that were used forthe first computational model, but instead for the second computationalmodel. In the present example embodiment, the hash of the secondcomputational model is created by hashing the compiled object code ofthe second computational model using an MD5 hashing algorithm, asdiscussed above in relation to hashing the first computational model.

In operation 308, computational determination/validation mod 404 sends arecord of the first computational determination to a verificationsystem, in this case blockchain mod 402. Generally speaking, the purposeof computational determination/validation mod 404 sending the record ofthe first computational determination to blockchain mod 402 is twofold:(i) for blockchain mod 402 to verify, using the first distributedledger, the computational model that was used to produce the firstcomputational determination, and (ii) to document information pertainingto the first computational determination for later review/analysis. Assuch, the record of the first computational determination includes atleast the hash of the second computational model, and in manycases—including the present example embodiment—the record also includesthe first input and the first computational model output.

In operation 310, computational determination/validation mod 404receives a verification from the verification system (blockchain mod402) indicating that the hash of the first computational model matchesthe hash of the second computational model and that the record of thefirst computational determination has been stored in the seconddistributed ledger. In other words, computationaldetermination/validation mod 404 receives verification that the recordof the first computational determination has been matched to acomputational model and then immutably stored in the second distributedledger. Then, in operation 312, computational determination/validationmod 404, in response to receiving the verification, provides the firstcomputational output to the user.

While the foregoing description and examples relating to FIG. 3 describea process for producing and validating a single computationaldetermination, embodiments of the present invention recognize that thenetworked computers system 100 described herein is also capable ofproducing and validating many additional computational determinations.For example, in an embodiment, computational determination/validationmod 404 receives a second input from a user (whether the user describedabove in relation to operation 302 or a different user), produces asecond computational determination utilizing the second computationalmodel (or a different computational model), computes a hash, sends arecord, receives verification, and provides the second computationaldetermination to the user.

Certain embodiments of the present invention further provide a mechanismfor reviewing and/or auditing the production of certain computationaldeterminations made by computational determination/validation mod 404 ofnetworked computers system 100. In one of these embodiments,computational determination/validation mod 404 receives a request froman auditing entity to audit the production of the first computationaldetermination. Then, in response to receiving the request to audit theproduction of the first computational determination, computationaldetermination/validation mod 404 retrieves from blockchain mod 402 therecord of the first computational determination stored in the seconddistributed ledger, and provides at least a portion of the retrievedrecord of the first computational determination to the auditing entity.

The request to audit the production of the first computationaldetermination may identify the first computational determination in oneor more of several possible ways. For example, in some embodiments, therequest to audit the production of the first computational determinationidentifies the first computational determination using the first input.In other embodiments, the request to audit the production of the firstcomputational determination identifies the first computationaldetermination using the first computational output. In still otherembodiments, the request to audit the production of the firstcomputational determination identifies the first computationaldetermination using both the first input and the first computationaloutput.

In some embodiments, providing the at least a portion of the retrievedrecord of the first computational determination to the auditing entityincludes providing the first input and the first computational output tothe auditing entity. However, in other embodiments, the amount ofinformation provided to the auditing entity is limited based on ruleand/or policy. For example, in a situation where the first inputincludes personal information of an individual, computationaldetermination/validation mod 404 may provide a modified version of thefirst input to the auditing entity where all or some of the personalinformation has been redacted.

In some embodiments, responding to the request to audit the productionof the first computational determination further includes computationaldetermination/validation mod 404 further retrieving from blockchain mod402 the first computational model that matches the second computationalmodel, where blockchain mod 402 provides the stored copy of the firstcomputational model. In these embodiments, blockchain mod 402 determinesthe corresponding model to provide based on the record of the firstcomputational determination in the second distributed ledger. Oncecomputational determination/validation mod 404 has retrieved the firstcomputational model from blockchain mod 402, computationaldetermination/validation mod 404 provides the first computational modelto the auditing entity. Again, as with the information relating to thefirst computational determination, computationaldetermination/validation mod 404 may, based on rule and/or policy,provide only a portion of the first computational model to the auditingentity. For example, computational determination/validation mod 404 mayprovide access to the first computational model for providing additionalinputs/outputs but might not provide access to the first computationalmodel's underlying source code.

FIG. 5 illustrates computational determination environment 500, inaccordance with an embodiment of the present invention. As shown,computational determination environment 500 comprises a developmentenvironment 502, a deployment environment 504, a production environment506, an AI model blockchain 508, and an inference data blockchain 510.In this embodiment, artificial intelligence (AI) models (i.e.,computational models) are developed in development environment 502. Whenthe AI models are ready for deployment, they are moved to deploymentenvironment 504, where they are added to AI model blockchain 508 (i.e.,a first distributed ledger) according to methods described above. Oncethe AI models are added to AI model blockchain 508, they are moved toproduction environment 506. Then, in response to a user 512 providinguser input, the AI models generate AI inferences (i.e., computationaldeterminations) including inference results (i.e., computational modeloutput) based on model input (i.e., the user input), and the results ofthose inferences are added to inference data blockchain 510 (i.e., asecond distributed ledger). As shown, in accordance with how blockchainblocks are typically structured, each block in AI model blockchain 508and in inference data blockchain 510 includes the hash of the blockpreceding it in the blockchain, where applicable. Further, as describedabove, each inference added to inference data blockchain 510 includes ahash of an AI model that corresponds to the hash of an AI model in theAI model blockchain 508. Once the AI inferences are added to theinference data blockchain 510, the inference results are provided touser 512 as output 514.

Definitions

Present invention: should not be taken as an absolute indication thatthe subject matter described by the term “present invention” is coveredby either the claims as they are filed, or by the claims that mayeventually issue after patent prosecution; while the term “presentinvention” is used to help the reader to get a general feel for whichdisclosures herein are believed to potentially be new, thisunderstanding, as indicated by use of the term “present invention,” istentative and provisional and subject to change over the course ofpatent prosecution as relevant information is developed and as theclaims are potentially amended.

Embodiment: see definition of “present invention” above—similar cautionsapply to the term “embodiment.”

and/or: inclusive or; for example, A, B “and/or” C means that at leastone of A or B or C is true and applicable.

Including/include/includes: unless otherwise explicitly noted, means“including but not necessarily limited to.”

Module/Sub-Module: any set of hardware, firmware and/or software thatoperatively works to do some kind of function, without regard to whetherthe module is: (i) in a single local proximity; (ii) distributed over awide area; (iii) in a single proximity within a larger piece of softwarecode; (iv) located within a single piece of software code; (v) locatedin a single storage device, memory or medium; (vi) mechanicallyconnected; (vii) electrically connected; and/or (viii) connected in datacommunication.

Computer: any device with significant data processing and/or machinereadable instruction reading capabilities including, but not limited to:desktop computers, mainframe computers, laptop computers,field-programmable gate array (FPGA) based devices, smart phones,personal digital assistants (PDAs), body-mounted or inserted computers,embedded device style computers, application-specific integrated circuit(ASIC) based devices.

What is claimed is:
 1. A computer-implemented method comprising:receiving a first input from a user; producing a first computationaldetermination utilizing a first computational model deployed in aproduction environment, wherein the first computational determinationincludes a first computational output that is based, at least in part,on the first input; computing a hash of the first computational modelusing a cryptographic hash function; sending a record of the firstcomputational determination to a verification system, wherein the recordof the first computational determination includes the hash of the firstcomputational model; receiving a verification from the verificationsystem indicating that the hash of the first computational model matchesa hash of a second computational model and that the record of the firstcomputational determination has been stored in a first distributedledger; and in response to receiving the verification, providing thefirst computational output to the user.
 2. The computer-implementedmethod of claim 1, wherein the first computational model is a machinelearning model.
 3. The computer-implemented method of claim 1, whereinthe record of the first computational determination further includes thefirst input and the first computational output.
 4. Thecomputer-implemented method of claim 3, further comprising: receiving areceiving a request from an auditing entity to audit the production ofthe first computational determination; in response to receiving therequest to audit the production of the first computationaldetermination, retrieving from the verification system the record of thefirst computational determination stored in the first distributedledger; and providing at least a portion of the retrieved record of thefirst computational determination to the auditing entity.
 5. The methodof claim 4, wherein the request to audit the production of the firstcomputational determination includes an identification selected from thegroup consisting of: the first input and the first computational output.6. The method of claim 4, wherein providing the at least a portion ofthe retrieved record of the first computational determination to theauditing entity includes providing the first input and the firstcomputational output to the auditing entity.
 7. The method of claim 4,further comprising: in response to receiving the request to audit theproduction of the first computational determination, further retrievingfrom the verification system the second computational model that matchesthe first computational model, wherein the second computational model isstored on a second distributed ledger; and providing the secondcomputational model to the auditing entity.
 8. A computer programproduct comprising a computer readable storage medium having programinstructions embodied therewith, wherein the computer readable storagemedium is not a transitory signal per se, the program instructionsexecutable by a processor to cause the processor to perform a methodcomprising: receiving a first input from a user; producing a firstcomputational determination utilizing a first computational modeldeployed in a production environment, wherein the first computationaldetermination includes a first computational output that is based, atleast in part, on the first input; computing a hash of the firstcomputational model using a cryptographic hash function; sending arecord of the first computational determination to a verificationsystem, wherein the record of the first computational determinationincludes the hash of the first computational model; receiving averification from the verification system indicating that the hash ofthe first computational model matches a hash of a second computationalmodel and that the record of the first computational determination hasbeen stored in a first distributed ledger; and in response to receivingthe verification, providing the first computational output to the user.9. The computer program product of claim 8, wherein the firstcomputational model is a machine learning model.
 10. The computerprogram product of claim 8, wherein the record of the firstcomputational determination further includes the first input and thefirst computational output.
 11. The computer program product of claim10, wherein the method further comprises: receiving a receiving arequest from an auditing entity to audit the production of the firstcomputational determination; in response to receiving the request toaudit the production of the first computational determination,retrieving from the verification system the record of the firstcomputational determination stored in the first distributed ledger; andproviding at least a portion of the retrieved record of the firstcomputational determination to the auditing entity.
 12. The computerprogram product of claim 11, wherein the request to audit the productionof the first computational determination includes an identificationselected from the group consisting of: the first input and the firstcomputational output.
 13. The computer program product of claim 11,wherein providing the at least a portion of the retrieved record of thefirst computational determination to the auditing entity includesproviding the first input and the first computational output to theauditing entity.
 14. The computer program product of claim 11, whereinthe method further comprises: in response to receiving the request toaudit the production of the first computational determination, furtherretrieving from the verification system the second computational modelthat matches the first computational model, wherein the secondcomputational model is stored on a second distributed ledger; andproviding the second computational model to the auditing entity.
 15. Acomputer system comprising: one or more processors; and a computerreadable storage medium wherein: the one or more processors arestructured, located, connected, and/or programmed to execute programinstructions stored on the computer readable storage medium; and theprogram instructions, when executed by the one or more processors, causethe one or more processors to perform a method comprising: receiving afirst input from a user; producing a first computational determinationutilizing a first computational model deployed in a productionenvironment, wherein the first computational determination includes afirst computational output that is based, at least in part, on the firstinput; computing a hash of the first computational model using acryptographic hash function; sending a record of the first computationaldetermination to a verification system, wherein the record of the firstcomputational determination includes the hash of the first computationalmodel; receiving a verification from the verification system indicatingthat the hash of the first computational model matches a hash of asecond computational model and that the record of the firstcomputational determination has been stored in a first distributedledger; and in response to receiving the verification, providing thefirst computational output to the user.
 16. The computer system of claim15, wherein the first computational model is a machine learning model.17. The computer system of claim 15, wherein the record of the firstcomputational determination further includes the first input and thefirst computational output.
 18. The computer system of claim 17, whereinthe method further comprises: receiving a receiving a request from anauditing entity to audit the production of the first computationaldetermination; in response to receiving the request to audit theproduction of the first computational determination, retrieving from theverification system the record of the first computational determinationstored in the first distributed ledger; and providing at least a portionof the retrieved record of the first computational determination to theauditing entity.
 19. The computer system of claim 18, wherein therequest to audit the production of the first computational determinationincludes an identification selected from the group consisting of: thefirst input and the first computational output.
 20. The computer systemof claim 18, wherein providing the at least a portion of the retrievedrecord of the first computational determination to the auditing entityincludes providing the first input and the first computational output tothe auditing entity.